BY  ORDER  OF  THE 
SECRETARY  OF  THE  AIR  FORCE 


22  JULY  1993 
AIR  FORCE  POLICY  DIRECTIVE  99-1 


19971105  032 


Test  and  Evaluation 
TEST  AND  EVALUATION  PROCESS 


1.1.  The  Air  Force  needs  to  ensure  the  weapons  systems  it  uses,  and  the  systems  that  support  them,  meet  or  exceed 
operational  requirements  in  terms  of  effectiveness  and  suitability  and  are  ready  for  fielding.  To  do  so,  the  Air  Force  conducts 
realistic,  cost-effective,  and  credible  test  and  evaluation  (T&E)  programs  through  all  phases  of  the  acquisition  and  fielding 
process.  This  directive  establishes  policies  for  the  T&E  process  and  infrastructure. 


1.2.  The  Air  Force  (T&E)  process  will: 

1.2.1.  Through  Developmental  Test  and  Evaluation  (DT&E),  Qualification  Test  and  Evaluation  (QT&E),  including 
Contractor  Testing  (CT),  demonstrate  systems  feasibility,  confirm  engineering  design  and  development  are  complete, 
minimize  design  risks,  and  ensure  systems  perform  as  required  in  their  intended  environments.  DT&E  and  QT&E  results 
support  weapon  system  life  cycle  decisions  and  certification  of  the  system  as  ready  for  dedicated  Air  Force  Initial 
Operational  Test  and  Evaluation  (IOT&E). 

1.2.2.  Through  IOT&E,  Qualification  Operational  Test  and  Evaluation  (QOT&E),  or  Follow-on  Test  and  Evaluation 
(FOT&E),  demonstrate,  under  as  operationally  realistic  conditions  as  possible  and  practical,  that  systems  are  operationally 
effective,  suitable,  and  capable  of  meeting  the  user’s  requirements.  IOT&E  and  QOT&E  results  support  the  decision  to 
proceed  beyond  low-rate  initial  production  or  to  field  the  system.  FOT&E,  including  Weapon  System  Evaluation  Programs 
(WSEP),  supports  decisions  relative  to  the  service  life  of  the  system. 

1.2.3.  The  Air  Force  T&E  process  will  not  develop  operational  requirements,  but  will  ensure  user-provided  requirements  can 
be  evaluated.  Emphasis  should  be  placed  on  key  parameters  and  threshold  values  found  in  the  Operational  Requirements 
Document  to  ensure  these  requirements  can  be  measured  and  rated  (AFPD  10-6,  Mission  Needs  and  Operational 
Requirements). 

1.3.  The  Air  Force  will  operate,  maintain,  and  improve  T&E  facilities,  including  Major  Range  and  Test  Facility  Bases 
(MRTFB)  and  the  US  Air  Force  Test  Pilot  School,  to  support  US  Air  Force,  Department  of  Defense,  and  other  valid  user 
requirements.  In  addition,  the  Air  Force  will  continually  evaluate  test  infrastructure  requirements  and  capabilities  to  ensure 
essential  Air  Force  needs  are  met,  and  to  avoid  unnecessary  duplication  and  expense  of  excess  test  resources. 

1.4.  The  Air  Force  will  conduct  WSEP  to  provide  ongoing,  realistic,  rigorous  T&E  of  complex  weapon  systems.  WSEP 
ensures  proper  weapon  system  performance  in  combat  and  supports  US  Air  Force  decision-makers  regarding  operational 
capabilities,  force  structure,  required  modifications,  and  requirements. 

1.5.  This  directive  establishes  the  following  responsibilities  and  authorities: 

1.5.1.  HQ  USAF/TE  is  responsible  for  policy,  advocacy  of  test  resources,  and  oversight  of  the  entire  T&E  process 
throughout  the  Air  Force. 

1.5.2.  Major  command  (MAJCOM),  field  operating  agency  (FOA),  Air  Reserve  components  (ARC),  direct  reporting  units 
(DRU),  and  other  center  commanders  ensure  T&E  programs  are  executed  in  compliance  with  this  policy.  Commanders  are 
responsible  for  maintaining  test  infrastructure  to  support  the  Air  Force  test  process,  including  both  DT&E  and  OT&E. 


1.6. 

1.7. 


This  policy  applies  to  all  MAJCOMs,  FOAs,  DRUs,  and  Air  Force  activ  ities,  including  ARC  . 


See  attachment  1  for  measures  of  compliance. 


1.8.  See  attachment  2  for  related  abbreviations,  acronyms,  and  terms. 
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1.9.  See  attachment  3  for  related  publications. 


HOWARD  W.  LEAF,  Lt  General,  USAF,  Retired 
Director  of  Test  and  Evaluation 
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MEASURING  AND  DISPLAYING  COMPLIANCE  WITH  POLICY 

ALL  Compliance  with  T&E  policy  will  be  assessed  quarterly  by  taking  measurements  related  to  two  major  areas: 

1)  Developmental  Test  and  Evaluation  (DT&E),  including  Contractor  Testing  (CT). 

2)  Operational  Test  and  Evaluation  (OT&E). 

Measurement  charts  will  display  trends  toward  improving  the  quality,  effectiveness,  and  efficiency  of  both  the  DT&E  and 
OT&E  programs. 

A1.1.L  The  policy  on  conducting  DT&E  or  QT&E  will  be  assessed  by  measuring  the  percentage  of  certified  programs 
failing  OT&E  (figure  Al.l).  Measurements  will  be  taken  by  HQ  AFMC/DO  and  reported  to  HQ  USAF/TE  no  later  than  10 
days  following  the  end  of  the  quarter.  HQ  USAF/TE  will  send  compiled  results  to  the  Chief  of  Staff  via  RCS:HAF-TEP(Q) 
9331,  Management  Information  System-Test  and  Evaluation  Brief.  The  report  is  designated  emergency  status  Code  D; 
discontinue  during  emergency  conditions. 

Al.1.2.  The  policy  on  conducting  IOT&E  and  QOT&E  will  be  assessed  by  measuring  the  percentage  of  final  OT&E  test 
reports  rejected  by  the  appropriate  Designated  Acquisition  Authority.  A  final  report  will  be  judged  deficient  if  either 
additional  operational  testing  is  required  as  a  result  of  inadequate  testing,  or  the  results  of  the  operational  test  are  challenged 
because  of  questionable  test  techniques  (figure  A1.2).  Measurements  will  be  taken  by  HQ  Air  Force  Operational  Test 
Evaluation  Center  and  reported  to  HQ  USAF/TE  no  later  than  10  days  following  the  end  of  the  quarter.  HQ  USAF/TE  will 
send  compiled  results  to  the  Chief  of  Staff  via  RCS:HAF-TEP(Q)  9332,  Management  Information  System-Test  and 
Evaluation  Brief.  This  report  is  designated  emergency  status  Code  D;  discontinue  during  emergency  conditions. 
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Figure  Al.l.  Sample  Metric  of  Certified  Programs  Failing  OT&E. 


OT&E  Results  Rejected  by  DAC 
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Figure  A1.2.  Sample  Metric  of  OT&E  Results  Rejected  by  DAC. 
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ABBREVIATIONS,  ACRONYMS,  AND  TERMS 

A2.1.  Developmental  Test  and  Evaluation  (DT&E),  including  contractor  testing,  is  conducted  to  evaluate  design  approaches, 
validate  analytical  models,  quantify  contract  technical  performance  and  manufacturing  quality,  measure  progress  in  system 
engineering  design  and  development,  minimize  design  risks,  and  predict  integrated  system  operational  performance 
(effectiveness  and  suitability)  in  the  intended  environment  and  identify  system  problems  (or  deficiencies)  to  allow  for  early 
and  timely  resolution  or  correction. 

A.2.2.  Follow-on  Operational  Test  and  Evaluation  (FOT&E)  is  used  to  refine  the  estimates  made  during  initial  operational 
test  and  evaluation  or  qualification  operational  test  and  evaluation,  evaluate  changes  and  verify  correction  of  deficiencies, 
assist  in  tactics  development,  and  to  reevaluate  the  system  to  ensure  that  it  continues  to  meet  operational  needs  and  retains  its 
effectiveness  and  suitability  in  a  new  environment  or  against  a  new  threat.  FOT&E  is  normally  conducted  by  the  using 
command  unless  higher  headquarters  directs  otherwise. 

A2.3.  Initial  Operational  Test  and  Evaluation  (IOT&E)  is  conducted  under  conditions  that  are  operationally  realistic  as 
possible  and  practical,  using  production  or  production  representative  test  articles  to  determine  if  the  system,  as  developed, 
will  meet  the  user’s  defined  operational  requirements.  IOT&E  can  also  be  used  to  assist  the  using  command  in  tactics 
development. 

A2.4.  Major  Range  and  Test  Facility  Bases  (MRTFB)  are  those  specified  test  activities  defined  in  Department  of  Defense 
(DoD)  Directive  3200.11  as  "national  assets."  The  DoD  MRTFB  is  maintained  primarily  to  support  the  test  and  evaluation 
process  (developmental  test  and  operational  test)  of  US  military  systems.  In  addition,  many  of  the  MRTFB  test  resources 
(sites  and  testing  assets  referred  to  as  MRTFB  activities)  provide  a  training  capability. 

A2.5.  Qualification  Operational  Test  and  Evaluation  (QOT&E)  is  conducted  in  lieu  of  initial  operational  test  and  evaluation 
on  programs  for  which  there  has  been  no  research  and  development  effort,  and  on  certain  modifications  to  existing  systems. 

A2.6.  Qualification  Test  and  Evaluation  (QT&E)  is  conducted  in  lieu  of  developmental  test  and  evaluation,  which  may 
involve  contractor  and  government  testing,  on  programs  where  there  has  not  been  a  preceding  research  and  development 
effort  and  on  certain  modifications  to  existing  systems. 

A2.7.  Test  Resources  encompass  all  resources  needed  to  support  the  entire  test  and  evaluation  process,  including  but  not 
limited  to:  contractor  test  facilities,  test  ranges,  manpower,  training,  base  operations  support  (BOS),  flying  time,  support 
equipment,  threat  systems,  models  and  simulations,  instrumentation,  communications,  range  equipment  facilities,  data 
protection  and  security  systems,  and  munitions  and  targets. 
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RELATED  PUBLICATIONS  AND  INSTRUCTIONS 


Section  A— Implementing  Publications 


Publication  Number 

Publication  Title 

Publication  Date 

Title  10,  U.S.  Code, 

Section  2399 

Operational  Test  and  Evaluation  of 
Defense  Acquisition  Programs 

1986 

DoD  Directive  5000.1 

Defense  Acquisition 

February  23,  1991 

DoD  Instruction  5000.2, 

Part  7,  With  Change  1 

Defense  Acquisition  Management 
Policies  and  Procedures 

February  23,  1991 

DoD  5000.2-M,  Part  7, 

With  Change  1 

Defense  Acquisition  Management 
Documentation  and  Reports 

February  1991 

DoD  Directive  3200.1 1, 

With  Changes  1 

Through  3 

Major  Range  and  Test  Facility  Base 

September  29,  1980 

Section  B-Interfacing  Publications 

Publication  Number 

Publication  Title 

Formerly 

AH  99-101 

Developmental  Test  and  Evaluation 

AFRs  80-14  and  80-19 

AH  99-102 

Operational  Test  and  Evaluation 

AFRs  55-43  and  80-14 

AH  99-103 

Electronic  Combat  (EC)  Test  Process 

No  Former  Publication 

AH  99-104 

Software  Test  Process 

No  Former  Publication 

AH  99-105 

Live  Fire  Test 

No  Former  Publication 

AH  99-106 

Joint  Test  and  Evaluation 

AFR  80-20 

AH  99-107 

US  Air  Force  Test  Pilot  School 

AFR  53-19 
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Publication  Number 

Publication  Title 

Formerly 

AH  99-108 

Programming  and  Reporting  Missile  and 
Target  Expenditures  in  Test  and 
Evaluation 

AFR  55-11 

AH  99-109 

Test  Resource  Planning 

AFR  80-14 

AH  99-111 

Test  Infrastructure 

AFR  80-28 

AH  99-112 

System  Deficiency  Reporting 

NoFormerPublication 
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